{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numpy\n",
    "\n",
    "Python作为深度学习最主流的编程语言，在深度学习工程具体实现中提供了Numpy库(Numerical Python extensions)来作为工具，帮助使用者实现深度学习神经网络，甚至用于实现深度学习框架。\n",
    "\n",
    "Numpy是一个第三方的Python包，用于科学计算。这个库的前身是1995年就开始开发的一个用于数组运算的库。经过了长时间的发展，基本上成了绝大部分Python科学计算的基础包，当然也包括所有提供Python接口的深度学习框架。\n",
    "\n",
    "## 基本模块\n",
    "\n",
    "** Array模块 **\n",
    "\n",
    "Array，也就是数组，是numpy中最基础的数据结构。最关键的属性是维度和元素类型，在numpy中，可以非常方便地创建各种不同类型的多维数组，并且执行一些基本操作。在深度学习中，如果神经元之间的连接关系涉及到的参数是数组，便可利用Array模块进行设定，来看例子：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.5"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "a = [1, 2, 3, 4]     \t# a是python中的list类型\n",
    "b = np.array(a)         # 数组化之后b的类型变为 array\n",
    "type(b)                  # b的类型 <type 'numpy.ndarray'>\n",
    "\n",
    "b.shape                   # shape参数表示array的大小，这里是(4,)\n",
    "b.argmax()               # 调用max()函数可以求得array中的最大值的索引，这里是3\n",
    "b.max()                   # 调用max()函数可以求得array中的最大值，这里是4\n",
    "b.mean()                  # 调用mean()函数可以求得array中的平均值，这里是2.5\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意到在导入numpy的时候，代码中将np作为numpy的别名。这是一种习惯性的用法，后面的章节中本书也默认这么使用。例如在机器学习中常用到的矩阵的转置操作，可以通过matrix构建矩阵，transpose函数来实现转置：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x:\n",
      " [[ 0  1  2  3]\n",
      " [ 4  5  6  7]\n",
      " [ 8  9 10 11]]\n",
      "transposed x:\n",
      " [[ 0  4  8]\n",
      " [ 1  5  9]\n",
      " [ 2  6 10]\n",
      " [ 3  7 11]]\n"
     ]
    }
   ],
   "source": [
    "from numpy import matrix\n",
    "\n",
    "x=matrix(np.arange(12).reshape((3,4)))\n",
    "print(\"x:\\n\", x)\n",
    "t = x.transpose()\n",
    "print(\"transposed x:\\n\", t)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于一维的array，所有Python列表（list）支持的下标操作方法array也都支持，所以在此没有特别列出。\n",
    "既然叫numerical python，numpy本身所具有的基础数学运算也是强大的：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a: 1\n",
      "b: 1.0\n",
      "c: 0.500001071578\n",
      "d: 20.0855369232\n",
      "f: 8\n",
      "g: 11\n",
      "h: 5.0\n",
      "l: 10\n",
      "m: 5.5\n",
      "p: 0.968245836552\n"
     ]
    }
   ],
   "source": [
    "# 绝对值，1\n",
    "a = np.abs(-1)\n",
    "print(\"a:\", a)\n",
    "\n",
    "# sin函数，1.0\n",
    "b = np.sin(np.pi/2)\n",
    "print(\"b:\", b)\n",
    "\n",
    "# tanh逆函数，0.50000107157840523\n",
    "c = np.arctanh(0.462118)\n",
    "print(\"c:\", c)\n",
    "\n",
    "# e为底的指数函数，20.085536923187668\n",
    "d = np.exp(3)\n",
    "print(\"d:\", d)\n",
    "\n",
    "# 2的3次方，8\n",
    "f = np.power(2, 3)\n",
    "print(\"f:\", f)\n",
    "\n",
    "# 点乘，1*3+2*4=11\n",
    "g = np.dot([1, 2], [3, 4])\n",
    "print(\"g:\", g)\n",
    "\n",
    "# 开方，5\n",
    "h = np.sqrt(25)\n",
    "print(\"h:\", h)\n",
    "\n",
    "# 求和，10\n",
    "l = np.sum([1, 2, 3, 4])\n",
    "print(\"l:\", l)\n",
    "\n",
    "# 平均值，5.5\n",
    "m = np.mean([4, 5, 6, 7])\n",
    "print(\"m:\", m)\n",
    "\n",
    "# 标准差，0.96824583655185426\n",
    "p = np.std([1, 2, 3, 2, 1, 3, 2, 0])\n",
    "print(\"p:\", p)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Random模块 **\n",
    "\n",
    "numpy中的随机模块包含了随机数产生和统计分布相关的基本函数。Python本身也有随机模块random，不过numpy的random功能更丰富，随机模块一般会用于深度学习中的一些随机数的生成，seed的生成以及初始值的设定，具体的用法请看下列代码： \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'\\xfc\\x18\\xcf\\x01\\xeb\\xf2\\x93nX'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy.random as random\n",
    "\n",
    "# 设置随机数种子\n",
    "random.seed(42)\n",
    "\n",
    "# 产生一个1x3，[0,1)之间的浮点型随机数\n",
    "# array([[ 0.37454012,  0.95071431,  0.73199394]])\n",
    "# 后面的例子就不在注释中给出具体结果了\n",
    "random.rand(1, 3)\n",
    "\n",
    "# 产生一个[0,1)之间的浮点型随机数\n",
    "random.random()\n",
    "\n",
    "# 从a中有放回的随机采样7个\n",
    "a = np.array([1, 2, 3, 4, 5, 6, 7])\n",
    "random.choice(a, 7)\n",
    "\n",
    "# 从a中无放回的随机采样7个\n",
    "random.choice(a, 7, replace=False)\n",
    "\n",
    "# 对a进行乱序并返回一个新的array\n",
    "b = random.permutation(a)\n",
    "   \n",
    "# 生成一个长度为9的随机bytes序列并作为str返回\n",
    "# '\\x96\\x9d\\xd1?\\xe6\\x18\\xbb\\x9a\\xec'\n",
    "random.bytes(9)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 广播机制\n",
    "\n",
    "对于array，默认执行对位运算。涉及到多个array的对位运算需要array的维度一致，如果一个array的维度和另一个array的子维度一致，则在没有对齐的维度上分别执行对位运算，这种机制叫做广播（broadcasting），言语解释比较难，还是看例子理解："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 3,  4,  5],\n",
       "       [ 6,  7,  8],\n",
       "       [ 9, 10, 11],\n",
       "       [12, 13, 14]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "a = np.array([\n",
    "    [1, 2, 3],\n",
    "    [4, 5, 6]\n",
    "])\n",
    "\n",
    "b = np.array([\n",
    "    [1, 2, 3],\n",
    "    [1, 2, 3]\n",
    "])\n",
    "\n",
    "'''\n",
    "维度一样的array，对位计算\n",
    "array([[2, 4, 6],\n",
    "       [5, 7, 9]])\n",
    "'''\n",
    "a + b\n",
    "\n",
    "c = np.array([\n",
    "    [1, 2, 3],\n",
    "    [4, 5, 6],\n",
    "    [7, 8, 9],\n",
    "    [10, 11, 12]\n",
    "])\n",
    "d = np.array([2, 2, 2])\n",
    "\n",
    "'''\n",
    "广播机制让计算的表达式保持简洁\n",
    "d和c的每一行分别进行运算\n",
    "array([[ 3,  4,  5],\n",
    "       [ 6,  7,  8],\n",
    "       [ 9, 10, 11],\n",
    "       [12, 13, 14]])\n",
    "'''\n",
    "c + d\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 向量化\n",
    "\n",
    "向量化在深度学习中的应用十分广泛，它是提升计算效率的主要手段之一。对于在机器学习中缩短每次训练的时间是很有意义的，当可用工作时间不变的情况下，更短的单次训练时间可以让程序员有更多的测试机会，进而更早更好的调整神经网络结构和参数。接下来通过一个矩阵相乘的例子来呈现向量化对于代码计算速度的提升效果。\n",
    "\n",
    "首先导入了numpy和time库，它们分别被用于数学计算和统计运行时间。然后准备数据，这里初始化两个1000000维的随机向量v1和v2，v作为计算结果初始化为零。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import time\n",
    "#初始化两个1000000维的随机向量v1,v2用于矩阵相乘计算\n",
    "v1 = np.random.rand(1000000)\t\n",
    "v2 = np.random.rand(1000000)\n",
    "v = 0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设置变量tic和toc分别为计算开始时间和结束时间。在非向量化版本中，两个向量相乘的计算过程使用for循环实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "非向量化-计算结果：249918.031243ms\n",
      "非向量化-计算时间：549.713134765625ms\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#矩阵相乘-非向量化版本\n",
    "tic = time.time()\n",
    "for i in range(1000000):\n",
    "    v += v1[i]*v2[i]\n",
    "toc = time.time()\n",
    "print(\"非向量化-计算结果：\" + str(v) + \"ms\")\n",
    "print(\"非向量化-计算时间：\" + str((toc - tic)*1000)+\"ms\"+\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样使用变量tic和toc记录计算开始和结束时间。向量化版本使用numpy库的numpy.dot()计算矩阵相乘。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "向量化-计算结果：249918.031243ms\n",
      "向量化-计算时间：4.447221755981445ms\n"
     ]
    }
   ],
   "source": [
    "#矩阵相乘-向量化版本\n",
    "tic = time.time()\n",
    "v = np.dot(v1, v2)\n",
    "toc = time.time()\n",
    "print(\"向量化-计算结果：\" + str(v) + \"ms\")\n",
    "print(\"向量化-计算时间：\" + str((toc - tic)*1000)+\"ms\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以观察到效率提升效果十分显著。非向量化版本的计算时间约为向量化版本计算时间的500倍。可见向量化对于计算速度的提升是很明显的，尤其是在长时间的深度学习训练中，向量化可以帮助开发者节省更多时间。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
